5 research outputs found
Audio-based Roughness Sensing and Tactile Feedback for Haptic Perception in Telepresence
Haptic perception is highly important for immersive teleoperation of robots,
especially for accomplishing manipulation tasks. We propose a low-cost haptic
sensing and rendering system, which is capable of detecting and displaying
surface roughness. As the robot fingertip moves across a surface of interest,
two microphones capture sound coupled directly through the fingertip and
through the air, respectively. A learning-based detector system analyzes the
data in real time and gives roughness estimates with both high temporal
resolution and low latency. Finally, an audio-based vibrational actuator
displays the result to the human operator. We demonstrate the effectiveness of
our system through lab experiments and our winning entry in the ANA Avatar
XPRIZE competition finals, where briefly trained judges solved a
roughness-based selection task even without additional vision feedback. We
publish our dataset used for training and evaluation together with our trained
models to enable reproducibility of results.Comment: IEEE International Conference on Systems, Man, and Cybernetics (SMC),
Honolulu, Hawaii, USA, October 202
Robust Immersive Telepresence and Mobile Telemanipulation: NimbRo wins ANA Avatar XPRIZE Finals
Robotic avatar systems promise to bridge distances and reduce the need for
travel. We present the updated NimbRo avatar system, winner of the $5M grand
prize at the international ANA Avatar XPRIZE competition, which required
participants to build intuitive and immersive robotic telepresence systems that
could be operated by briefly trained operators. We describe key improvements
for the finals, compared to the system used in the semifinals: To operate
without a power- and communications tether, we integrated a battery and a
robust redundant wireless communication system. Video and audio data are
compressed using low-latency HEVC and Opus codecs. We propose a new locomotion
control device with tunable resistance force. To increase flexibility, the
robot's upper-body height can be adjusted by the operator. We describe
essential monitoring and robustness tools which enabled the success at the
competition. Finally, we analyze our performance at the competition finals and
discuss lessons learned.Comment: M. Schwarz and C. Lenz contributed equall
FaDIV-Syn: Fast Depth-Independent View Synthesis
We introduce FaDIV-Syn, a fast depth-independent method for novel view
synthesis. Related methods are often limited by their depth estimation stage,
where incorrect depth predictions can lead to large projection errors. To avoid
this issue, we efficiently warp input images into the target frame for a range
of assumed depth planes. The resulting plane sweep volume (PSV) is directly fed
into our network, which first estimates soft PSV masks in a self-supervised
manner, and then directly produces the novel output view. We therefore
side-step explicit depth estimation. This improves efficiency and performance
on transparent, reflective, thin, and feature-less scene parts. FaDIV-Syn can
perform both interpolation and extrapolation tasks and outperforms
state-of-the-art extrapolation methods on the large-scale RealEstate10k
dataset. In contrast to comparable methods, it achieves real-time performance
due to its lightweight architecture. We thoroughly evaluate ablations, such as
removing the Soft-Masking network, training from fewer examples as well as
generalization to higher resolutions and stronger depth discretization